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Abstract 

The robust adaptive beamforming (RAB) problem for general-rank signal model with an additional 
positive semi-definite constraint is considered. Using the principle of the worst-case performance opti- 
mization, such RAB problem leads to a difference-of-convex functions (DC) optimization problem. The 
existing approaches for solving the resulted non-convex DC problem are based on approximations and 
find only suboptimal solutions. Here we solve the non-convex DC problem rigorously and give arguments 
suggesting that the solution is globally optimal. Particularly, we rewrite the problem as the minimization 
of a one-dimensional optimal value function whose corresponding optimization problem is non-convex. 
Then, the optimal value function is replaced with another equivalent one, for which the corresponding 
optimization problem is convex. The new one-dimensional optimal value function is minimized iteratively 
via polynomial time DC (POTDC) algorithm. We show that our solution satisfies the Karush-Kuhn-Tucker 
(KKT) optimality conditions and there is a strong evidence that such solution is also globally optimal. 
Towards this conclusion, we conjecture that the new optimal value function is a convex function. The 
new RAB method shows superior performance compared to the other state-of-the-art general-rank RAB 
methods. 



I. Introduction 

It is well known that when the desired signal is present in the training data, the performance of 
adaptive beamforming methods degrades dramatically in the presence of even a very slight mismatch in 
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the knowledge of the desired signal covariance matrix. The mismatch between the presumed and actual 
source covariance matrices occurs because of, for example, displacement of antenna elements, time 
varying environment, imperfections of propagation medium, etc. The main goal of any robust adaptive 
beamforming (RAB) technique is to provide robustness against any such mismatches. 

Most of the RAB methods have been developed for the case of point source signals when the rank 
of the desired signal covariance matrix is equal to one lITI- lfTOl . Among the principles used for such 
RAB methods design are i) the worst-case performance optimization IH-lSl; ii) probabilistic based 
performance optimization ||7]; and iii) estimation of the actual steering vector of the desired signal 
|[8l- |[T0l . In many practical appUcations such as, for example, the incoherently scattered signal source or 
source with fluctuating (randomly distorted) wavefronts, the rank of the source covariance matrix is higher 
than one. Although the RAB methods of |[ll- |[TOl provide excellent robustness against any mismatch of 
the underlying point source assumption, they are not perfectly suited to the case when the rank of the 
desired signal covariance matrix is higher than one. 

The RAB for the general-rank signal model based on the explicit modeling of the error mismatches has 
been developed in ifTTI based on the worst-case performance optimization principle. Although the RAB 
of ifTTI has a simple closed form solution, it is overly conservative because the worst-case correlation 
matrix of the desired signal may be negative-definite |[T2l - |[T4l . Thus, less conservative approaches have 
been developed in |[T2l - |[T4l by considering an additional positive semi-definite (PSD) constraint to the 
worst-case signal covariance matrix. The major shortcoming of the RAB methods of |[T2l - |[T4l is that they 
find only a suboptimal solution and there may be a significant gap to the global optimal solution. For 
example, the RAB of lfT2l finds a suboptimal solution in an iterative way, but there is no guarantee that 
such iterative method converges ||T4| . A closed-form approximate suboptimal solution is proposed in |[T3l . 
however, this solution may be quite far from the globally optimal one as well. All these shortcomings 
motivate us to look for new efficient ways to solve the aforementioned non-convex problem globally 
optimallyQ 

We propose a new method that is based on recasting the original non-convex difference-of-convex 
functions (DC) programming problem as the minimization of a one dimensional optimal value function. 
Although the corresponding optimization problem of the newly introduced optimal value function is non- 
convex, it can be replaced with another equivalent function. The optimization problem that corresponds to 
such new optimal value function is convex and can be solved efficiently. The new one-dimensional optimal 

'Some preliminary results have been presented in 0151 . 
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value function is then minimized by the means of the newly designed polynomial time DC (POTDC) 
algorithm (see also |[T6l . IITtI ). We prove that the point found by the POTDC algorithm for RAB for 
general-rank signal model with positive semi-definite constraint is a Karush-Kuhn-Tucker (KKT) optimal 
point. Moreover, we prove a number of results that lead us to the equivalence between the claim of global 
optimality for the POTDC algorithm as applied to the problem under consideration and the convexity of 
the newly obtained one-dimensional optimal value function. The latter convexity of the newly obtained 
one-dimensional optimal value function can be checked numerically by using the convexity on lines 
property of convex functions. The fact that enables such numerical check is that the argument of such 
optimal value function is proved to take values only in a closed interval. In addition, we also develop 
tight lower-bound for such optimal value function that is used in the simulations for further confirming 
global optimality of the POTDC method. 

The rest of the paper is organized as follows. System model and preliminaries are given in Section II, 
while the problem is formulated in Section III. The new proposed method is developed in Section IV 
followed by the simulation results in Section V. Finally, Section VI presents our conclusions. This paper 
is reproducible research and the software needed to generate the simulation results will be provided to 
the IEEE Xplore together with the paper upon its acceptance. 

II. System Model and Preliminaries 

The narrowband signal received by a linear antenna array with M omni-directional antenna elements 
at the time instant k can be expressed as 

^{k)=s{k) + i{k) + u{k) (1) 

where s(A;), i(A;), and n(A;) are the statistically independent M x 1 vectors of the desired signal, 
interferences, and noise, respectively. The beamformer output at the time instant k is given as 

y{k) = w^x(A:) (2) 

where w is the M x 1 complex beamforming vector of the antenna array and (•)^ stands for the 
Hermitian transpose. The beamforming problem is formulated as finding the beamforming vector w 
which maximizes the beamformer output signal-to-interference-plus-noise ratio (SINR) given as 

SINR=^^^ (3) 

where Rg = E{s{k)s{k)^} and Rj+n = E{{\{k) + n{k)){\{k) + n{k))^} are the desired signal and 
interference -plus-noise covariance matrices, respectively, and E{-} stands for the statistical expectation. 
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Depending on the nature of the desired signal source, its corresponding covariance matrix can be of 
an arbitrary rank, i.e., 1 < rankjRs} < M, where rank{-} denotes the rank operator. Indeed, in many 
practical applications, for example, in the scenarios with incoherently scattered signal sources or signals 
with randomly fluctuating wavefronts, the rank of the desired signal covariance matrix Rg is greater than 
one ifTTIl . The only particular case in which, the rank of Rg is equal to one is the case of the point source. 

The interference-plus-noise covariance matrix Ri+n is typically unavailable in practice and it is sub- 
stituted by the data sample covariance matrix 

1 ^ 

i=l 

where K is number of the training data samples. The problem of maximizing the SINR Q (here we 
always use sample matrix estimate R instead of Ri+n) is known as minimum variance distortionless 
response (MVDR) beamforming and can be mathematically formulated as 

minw^Rw s.t. w^RgW = 1. (5) 

w 

The solution to the MVDR beamforming problem ([5]l can be found as |[T1 

WSMI-MVDR = PjR'^Rs} (6) 

which is known as the sample matrix inversion (SMI) MVDR beamformer for general-rank signal model. 
Here P{-} stands for the principal eigenvector operator. 

In practice, the actual desired signal covariance matrix Rg is usually unknown and only its presumed 
value is available. The actual source correlation matrix can be modeled as Rg = Rs + Ai, where Ai and 
Rg denote an unknown mismatch and the presumed correlation matrices, respectively. It is well known 
that the MVDR beamformer is very sensitive to such mismatches ifTTI . RABs also address the situation 
when the sample estimate of the data covariance matrix dD is inaccurate (for example, because of small 
sample size) and R = R + A2, where A2 is an unknown mismatch matrix to the data sample covariance 
matrix. In order to provide robustness against the norm-bounded mismatches ||Ai|| < e and IIA2II < 7 
(here || • || denotes the Frobenius norm of a matrix), the RAB of lITTI uses the worst-case performance 
optimization principle of ||2l and finds the solution as 

w = P{(R + 7l)-i(Rs-eI)}. (7) 

Although the RAB of fill has a simple closed-form solution Q, it is overly conservative because the 
constraint that the matrix Rg + Ai has to be positive semi-definite (PSD) is not considered |fT2l . For 
example, the worst-case desired signal covariance matrix Rg — el in ([7]l can be indefinite or negative 
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definite if Rg is rank deficient. Indeed, in the case of incoherently scattered source, Rs has the following 
form Rs = (Tg /^^^2 C,{G)ai{9)ai^ {9)d9 , where C,{9) denotes the normalized angular power density, a"^ is 
the desired signal power, and a(0) is the steering vector towards direction 9. For a uniform angular power 
density on the angular bandwidth the approximate numerical rank of Rs is equal to {^/ir) ■ M lITSl . 
This leads to a rank deficient matrix Rs if the angular power density does not cover all the directions. 
Therefore, the worst-case covariance matrix Rs — d is indefinite or negative definite. Note that the 
worst-case data sample covariance matrix R + 7I is always positive definite. 

III. Problem Formulation 

Decomposing Rs as Rs = Q^Q, the RAB problem for a norm bounded-mismatch ||A|| < 77 to the 
matrix Q is given as |[T2]| 

min max w^(R + A2)w 

w ||A2||<7 

s.t. min w^(Q + A)^(Q + A)w>l. (8) 

llA|l<r? 

For every A in the optimization problem dSjl whose norm is less than or equal to rj, the expression 
w^(Q+A)^(Q+A)w > 1 represents a non-convex quadratic constraint with respect to w. Because there 
exists infinite number of mismatches A, there also exists infinite number of such non-convex quadratic 
constraints. By finding the minimum possible value of the quadratic term w^(Q+ A)^(Q+ A)w with 
respect to A for a fixed w, the infinite number of such non-convex quadratic constraints can be replaced 
with a single constraint. For this goal, we consider the following optimization problem 

inin w^(Q + A)^(Q + A)w 

s.t. ||Af <r/2 (9) 

where A is a Hermitian matrix. This problem is convex and its optimal value can be expressed as a 
function of as given by the following lemma. 

Lemma 1: The optimal value of the optimization problem (|9]l as a function of is equal to 

min w^(Q + A)^(Q+A)w 

II A|p<»;2 

(IIQwII -r?|| w||)2, IIQwII > r/||w|| ^^^^ 

0, otherwise. 

Proof: See Appendix, Subsection VII-A. ■ 
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The maximum of the quadratic temi w^(R + A2)w with respect to A2, IIA2II < 7 that appears in 
the the objective of the problem dH) can be easily derived as w^(R + 7l)w. It is obvious from (ITOl ) that 
the desired signal can be totally removed from the beamformer output if ||Qvv^|| < t?||w||. Based on the 
later fact, ||Qw|| — 7?||w|| should be greater than or equal to zero. For any such w, the new constraint in 
the optimization problem ^ can be expressed as ||Qw|| — ?7|| w|| > 1. Since ||Qw|| — rj\\ w|| > 1 also 
implies that ||Qw|| — ?7||w|| > 0, the RAB problem (HJl can be equivalently rewritten as 

mill w^(R + 7l)w 

w 

s.t. IIQwII - r/||w|| > 1 . (11) 

Due to the non-convex DC constraint, (fTTI ) is non-convex DC programming problem |[T6l . IITtI . DC 
optimization problems are believed to be NP-hard in general |[T9l . |[20l . There is a number of methods 
that can be applied to DC problems of type ([TTI ) in the literature. Among these methods are the generalized 
polyblock algorithm, the extended general power iterative (GPI) algorithm |[2TI . DC iteration-based 
method Il22l . etc. However, the existing methods do not guarantee to find the solution of (fTTI ). i.e., 
to converge to the global optimum of ([TTT i in polynomial time. This means that the problem ([TTI ) is 
NP-hard. The best what is possible to show, for example, for the DC iteration-based method is that it 
can find a KKT optimal point. The overall computational complexity of the DC iteration-based method 
can be, however, quite high because the number of iterations required to converge grows dramatically 
with the dimension of the problem. 

Recently, the problem (fTTI ) has also been suboptimally solved using an iterative semi-definite relaxation 
(SDR)-based algorithm in |[T2l which also does not result in the globally optimal solution and for which 
the convergence even to a KKT optimal point is not guaranteed. A closed-form suboptimal solution 
for the aforementioned non-convex DC problem has been also derived in [13]. Despite its computational 
simplicity, the performance of the method of llT3l may be far from the global optimum and even the KKT 
optimal point. Another iterative algorithm has been proposed in llT4l . but it modifies the problem (ITTI ) 
and solves the modified problem instead which again gives no guarantees for finding the globally optimal 
solution of the original problem dTTT l. In what follows, we develop a new polynomial time algorithm for 
addressing the DC programming problems of type (ITTT i. prove that it finds at least a KKT optimal point 
of the problem, and attempt to prove that it actually solves the problem. 
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IV. New Proposed Method 

In this section, we aim at solving the problem (fTTI) in a rigorous way, i.e., without using any type of 
approximations. For this goal, we design a POTDC-type algorithm (see also |[T6l . ifTTl ) that can be used 
for solving a class of DC programming problems in polynomial time. More specifically, the POTDC 
algorithm can be efficiently used for DC programming problems whose non-convex parts are functions 
of only one variable. By introducing the auxiliary optimization variable a > 1 and setting ||Qw|| = ^/a, 
the problem (fTTI) can be equivalently rewritten as 

mill w'^(R + 7l)w 

S.t. w^Q^Qw = a 

w^w< ^^7^^' , a>l. (12) 

Note that a is restricted to be greater than or equal to one because ||Qw|| is greater than or equal to 
one due to the constraint of the problem ([TTT i. For future needs, we find the set of all a's for which the 
optimization problem ([T2l ) is feasible. Let us define the following set for a fixed value of a > 1, 

5(a) = {w I w^w < (Va - (13) 

It is trivial that for every G S{a), the quadratic term mv^ Qw is non-negative as Q^Q is a positive 
semi-definite matrix. Using the minimax theorem ll23l . it can be easily verified that the maximum value 
of the quadratic term w^Q^Qw over w G S{a) is equal to [{^/a — Vf'/rf) ■ Amax{Q^Q} and this 
value is achieved by 

w« = ^^^P{Q^Q} G S{a). (14) 
V 

Here Amax{'} stands for the largest eigenvalue operator. Due to the fact that for any < /3 < 1, the 
scaled vector Pw^ lies inside the set S{a) ([T3] ). the quadratic term w^^Q^Qw can take values only in 
the interval [0, ((^a - 1)V??^) • Amax{Q^Q}] over w G S{a). 

Considering the later fact and also the optimization problem (fT2l) . it can be concluded that a is feasible 
if and only if a G [0, {{^/a - 1)V^^) • Amax{Q^Q}] which impl ies that 

^^7^^' •A^ax{Q^Q}>« (15) 

or, equivalently, that 

a Amax{Q^Q}" 

The function {^/a — is strictly increasing and it is also less than or equal to one for a > 1. 

Therefore, it can be immediately found that the problem (fTZt is infeasible for any a > 1 if Amax{Q^Q} < 
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rf'. Thus, hereafter, it is assumed that Amax{Q^Q} > ff- Moreover, using ([T6] l and the fact that the 
function (^/a — l)^/a is strictly increasing, it can be found that the feasible set of the problem (fT2]) 
corresponds to 

a > > 1. (17) 



As we will see in the following sections, for developing the POTDC algorithm for the problem ([T2] |. 
an upper-bound for the optimal value of a in (fT2]) is needed. Such upper-bound is obtained in terms of 
the following lemma. 

Lemma 2: The optimal value of the optimization variable a in the problem (fT2l ) is upper-bounded 
by Amax I (R- + 7I)^^Q'^Q| • Wo^(R + 'yl)-wo, where wq is any arbitrary feasible point of the problem 

m- 

Proof: See Appendix, Subsection VII-B. ■ 
Using Lemma 2, the problem ([T2] | can be equivalently stated as 

Inner Problem 



Mr\H, 



min minw (R + 7l)w 

9i<a<e2 w 



where 



s.t. w^Q^Qw=a, 

w^w< ^^ ^ ' (18) 



(19) 



VAmax{Q«Q} 



and 



A, 



'(R + 7l)-iQ^Q}w^(R + 7l)wo. (20) 

It is easy to verify that 02 > Oi. For a fixed value of a, the inner optimization problem in ([TsT i is non- 
convex with respect to w. Based on the inner optimization problem in ([T8] ) when a is fixed, we define 
the following optimal value function 



h{a) = [ minw^(R+7l)w | w^Q^Qw= a, 

^^|, ei<a<e2. (21) 



TT {\/a — l 
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Using the optimal value function (1211) . the problem (iTSI ) can be equivalently expressed as 

min h{a) s.t. Oi < a < 62- (22) 

a 

The corresponding optimization problem of h{a) for a fixed value of a is non-convex. In what follows, 
we aim at replacing h{a) with an equivalent optimal value function whose corresponding optimization 
problem is convex. 

Introducing the matrix W = v^v^^ and using the fact that for any arbitrary matrix A, w^Aw = 
tr{Aw^w^^} (here tr{-} stands for the trace of a matrix), the function (l2Ti can be equivalently recast as 

a, 

w 



h{a) = |mmtr{(R + 7l)W} | tr{Q^QW} 
tr{W}< ^^^T'^^ , W >i 0, ranklW} = l| 



1 < a < 02 (23) 



Dropping the rank-one constraint in the corresponding optimization problem of h{a) for a fixed value 
of a, {Oi < a < 62), a new optimal value function denoted as k{a) can be defined as 

k{a) = |miiitr{(R + 7l)W} I tr{Q^QW} = a, 

MW)<<^, wvo}. 

Oi<a< 62. (24) 

For brevity, we will refer to the optimization problems that correspond to the optimal value functions 
h{a) and k{a) when a is fixed, as the optimization problems of h{a) and k{a), respectively. Note also 
that compared to the optimization problem of h{a), the optimization problem of k{a) is convex. More 
importantly, the following lemma establishes the equivalence between the optimal value functions h{a) 
and k{a). 

Lemma 3: The optimal value functions h{a) and k{a) are equivalent, i.e., h{a) = k{a) for any 
a G [6'i,02]- Furthermore, based on the optimal solution of the optimization problem of k{a) when a is 
fixed, the optimal solution of the optimization problem of h{a.) can be constructed. 

Proof: See Appendix, Subsection VII-C. ■ 

Based on Lemma 3, the original problem (|22] | can be expressed as 



min k{a) s.t. 9i < a < 62 (25) 



It is noteworthy to mention that based on the optimal solution of (1251 ) denoted as aopt> we can easily 
obtain the optimal solution of the original problem (|22] | or, equivalently, the optimal solution of the 
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problem ([TSl l. Specifically, since the optimal value functions h{a) and k{a) are equivalent, Oopt is also 
the optimal solution of the problem (|22l) and, thus, also the problem ([TSl l. Moreover, the optimization 
problem of A:(aopt) is convex and can be easily solved. In addition, using the results in Lemma 3, based 
on the optimal solution of the optimization problem of /c(aopt), the optimal solution of the optimization 
problem of /i(aopt) can be constructed. Therefore, in the rest of the paper, we concentrate on the problem 



Since for every fixed value of a, the corresponding optimization problem of k{a) is a convex semi- 
definite programming (SDP) problem, one possible approach for solving (|25] ) is based on exhaustive 
search over a. In other words, a can be found by using an exhaustive search over a fine grid on the 
interval of [9i, 62]. Although this search method is inefficient, it can be used as a benchmark. 

Using the definition of the optimal value function k{a), the problem (|25] i can be equivalently expressed 

as 



Note that replacing ^.(q) by k{a) results in a much simpler problem. Indeed, compared to the original 
problem ([TST l. in which the first constraint is non-convex, the corresponding first constraint of (|26) is 
convex. All the constraints and the objective function of the problem (|26] l are convex except for the 
constraint tr{W} < {^/a — 1)^/??^ which is non-convex only in a single variable a and which makes 
the problem (l26l ) non-convex overall. This single non-convex constraint can be rewritten equivalently 
as ?7^tr{W} — (a + 1) + 2^/a < where all the terms are linear with respect to W and a except 
for the concave term of ^/a. The latter constraint can be handled iteratively by building a POTDC-type 
algorithm (see also |[T6l . lITTl ) based on the iterative linear approximation of the non-convex term ^/a 
around suitably selected points. It is interesting to mention that this iterative linear approximation can be 
also interpreted in terms of DC iteration over a single non-convex term ^/a. The fact that iterations are 
needed only over a single variable helps to reduce dramatically the number of iterations of the algorithm 
and allows for very simple interpretations shown below. 




s.t. tr{Q^QW} = a 

r|hr{W}<{^/Ti-lf 



W to, ei<a<92. 



(26) 
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A. Iterative POTDC Algorithm 

Let us consider the optimization problem (|26] | and replace the term ^/a by its linear approximation 
around ac, i.e., ^/a + (a — ac)/(2y^). It leads to the following SDP problem 

min tr((R + 7l)w| 
s.t. tr{Q^QW} = a 

r]hi{W} + {^c-l) + a(^--^^-lj <0 

W ^ 0, 6*1 < a < 6*2. (27) 

To understand the POTDC algorithm intuitively and also to see how the linearization points are selected in 
different iterations, let us define the following optimal value function based on the optimization problem 

l{a, ac) = { mhi tr { (R + 7I) W} | tr{Q^QW} = a, 
r/2tr{W} + (V^-l) + a (^-^-1^ <0, 

W>r0|, ei<a<e2. (28) 

where ac in l{a,ac) denotes the linearization point. The optimal value function l{a,ac) can be also 
obtained through k{a) in (l24l ) by replacing the term ^/a in 77^tr{W} — (a + 1) + 2y^ < with its linear 
approximation around ac- Since ^/a and its linear approximation have the same values at Oc, l{a,ac) 
and k{a) take the same values at this point. The following lemma establishes the relationship between 
the optimal value functions k{a) and l{a,ac)- 

Lemma 4: l{a,ac) is a convex upper-bound of k{a) for any arbitrary Oc G [6*1, ^2]. i.e., l{a,ac) > 
k{a), Va G [^1, 62] and l{a, ac) is convex with respect to a. Furthermore, the values of the optimal value 
functions k{a) and l{a,ac) as well as their right and left derivatives are equal at this point. In other 
words, under the condition that k{a) is differentiable at Oc, l{a,ac) is tangent to k{a) at this point. 
Proof: See Appendix, Subsection VII-D. ■ 

In what follows, for the sake of clarity of the explanations, it is assumed that the function k{a) is 
differentiable over the interval of (^1, 62), however, this property is not generally required as we will see 
later. Let us consider an arbitrary point, denoted as oq, oq G (^11^2) as the initial linearization point, 
i.e., ac = aQ. Based on Lemma 4, l{a,ao) is a convex function with respect to a which is tangent to 
k{a) at the linearization point a = qq, and it is also an upper-bound to k{a). Let ai denote the global 
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minimizer of l{a,ao) that can be easily obtained due to the convexity of l{a,ao) with polynomial time 
complexity. 

Since l{a, oq) is tangent to k{a) at a = oq and it is also an upper-bound for k{a), it can be concluded 
that ai is a decent point for k{a), i.e., k{ai) < k{ao) as it is shown in Fig{T] Specifically, the fact that 
l{a,ao) is tangent to k{a) at a = ao and ai is the global minimizer of l{a,ao) implies that 



l{ai,ao) < l{ao,ao) = k{ao) 



(29) 



Furthermore, since l{a,ao) is an upper-bound for k{a), it can be found that k{ai) < Z(ai,ao). Due to 
the later fact and also the equation ( [29l ). it is concluded that k{ai) < k{ao). 





a 






a 




1 ( 


a 


«i > 




Fig. 1. Iterative method for minimizing the optimal value function k{a). The convex optimal value function l{a,ao) is an 
upper bound to k{a) which is tangent to it at a = ao, and its minimum is denoted as ai. The point ai is used to establish 
another convex upper-bound function denoted as l{a,ai) and this proces continutes. 



Choosing ai as the linearzation point in the second iteration, and finding the global minimizer of 
l{a, ai) over the interval ^2] denoted as 02, another decent point can be obtained, i.e., k{a2) < k{ai). 
This process can be continued until convergence. 

Then the proposed iterative decent method can be described as shown in Algorithm 1. The following 
lemma about the convergence of Algorithm 1 and the optimality of the solution obtained by this algorithm 
is in order. Note that this lemma makes no assumptions about the differentiability of the optimal value 
function A; (a). 

Lemma 5: The following statements regarding Algorithm 1 are true: 

i) The optimal value of the optimization problem in Algorithm 1 is non-increasing over iterations. 
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I.e., 



tr{(R + 7l)Wopt,i+i} < tr{(R + 7l)Wopt,,} 



i > 1. 

ii) Algorithm 1 converges. 

iii) Algorithm 1 converges to a KKT solution, i.e., a solution which satisfies the KKT optimality 
conditions. 

Proof: See Appendix, Subsection VII-E. ■ 

Algorithm 1 The iterative POTDC algorithm 
Require: An arbitrary Oc € [^1,^2], 

the termination threshold (, 

set i equal to 1. 

repeat 

Solve the following optimization problem using Oc to obtain Wopt and aopt 

min tr |(R + 7l)w| 
S.t. tr{Q^QW} = a 

r?2tr{W} + (V^-l) + a(^-^-l^ <0 
W ^ 0, 6*1 < a < 6^2 

and set 

'^^opt,i ^ optt (^opt.i ^ ^opt 

"c ^ Oiopt, i ^ i + 1 

until 

tr |(R + 7l)Wopi,i_i}-tr{(R + 7l)Wopi,,} < C for i > 2 . 



There is a great evidence that the point obtained by Algorithm 1 (POTDC algorithm) is also the 
globally optimal point. This evidence is based on the following observation. The optimal value function 
k{a) of (l24l) is a convex function with respect to a. This observation is supported by numerous checks of 
convexity of k{a) for any arbitrary positive semi-definite matrices R and Rs = Q^Q. Such numerical 
checks are performed using the convexity on lines property of convex functions, and they are possible 
because a takes values only from a closed interval as it is shown before. As a result, if the optimal value 
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function k{a) of (l24l) is a convex function of a, the proposed POTDC algorithm achieves the global 
optimal solution (see Fig. 1 and corresponding explanations to how the POTDC algorithm works). The 
following formal conjecture is then in order. 

Conjecture 1: For any arbitrary positive semi-definite matrices R and Rg = Q^Q and positive values 
of 7 and r], the optimal value function k{a) defined in (l24l ) is a convex function of a G [^i, 6*2]. 

It is worth noting that even a more relaxed property of the optimal value function k{a) would be 
sufficient to guarantee global optimality for the POTDC algorithm. Specifically, if k{a) defined in (l24l ) 
is a strictly quasi-convex function of a € [9i, 62], then the point found by the POTDC algorithm will be 
the global optimum of the optimization problem ([TTI i. The evidence, however, is even more optimistic as 
stated in Conjecture 1 that (l24l ) is a convex function of a G [6*1,^2]- The computational complexity of 
Algorithm 1 is equal to that of the SDP optimization problem in Algorithm 1, that is, 0{{M + 1)^-^) times 
the number of iterations (see also Simulation example 1 in the next section). The RAB algorithm of |[T2l 
is iterative as well and its computational complexity is equal to 0{M^'^) times the number of iterations. 
The complexity of the RABs of ifTTl and |[T3l is 0{M^). The comparison of the overall complexity of 
the proposed POTDC algorithm with that of the DC iteration-based method will be explicitly performed 
in Simulation example 3. Although the computational complexity of the new proposed method may be 
slightly higher than that of the other RABs, it finds the global optimum and results in superior performance 
as it is shown in the next section. 

B. Lower-Bounds for the Optimal Value 

We also aim at developing a tight lower-bound for the optimal value of the optimization problem (|26] |. 
Such lower-bound can be used for assessing the performance of the proposed iterative algorithm. 

As it was mentioned earlier, although the objective function of the optimization problem (|26l ) is convex, 
its feasible set is non-convex due to the second constraint of (l26l) . A lower-bound for the optimal value 
of (l26l) can be achieved by replacing the second constraint of (l26l ) by its corresponding convex-hull. 
However, such lower-bound may not be tight. In order to obtain a tight lower-bound, we can divide the 
sector [^i,6'2] into N subsector and solve the optimization problem (|26] | over each subsector in which 
the second constraint of (l26l ) has been replaced with the corresponding convex hull. The minimum of 
the optimal values of such optimization problem over the sub sectors is the lower-bound for the problem 
(l26l) . It is obvious that by increasing A^, the lower-bound becomes tighter. 
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V. Simulation Results 

Let us consider a uniform linear array (ULA) of 10 omni-directional antenna elements with the 
inter-element spacing of half wavelength. Additive noise in antenna elements is modeled as spatially 
and temporally independent complex Gaussian noise with zero mean and unit variance. Throughout all 
simulation examples, it is assumed that in addition to the desired source, an interference source with the 
interference-to-noise ratio (INR) of 10 dB impinges on the antenna array. For obtaining each point in the 
simulation examples, 100 independent runs are used unless otherwise is specified and the sample data 
covariance matrix is estimated using K = 20 snapshots. 

The new proposed method is compared, in terms of the output SINR to the RAB methods of fTTl . 
|[T2l . and |[T3l . The proposed method, the method of |[T2l (the best among previous methods), and the 
lower-bound on the objective value of the problem ([TTI i are also compared in terms of the achieved values 
for the objective. 

The diagonal loading parameters of 7 = 10 and rj = 0.3ytr{R^ are chosen for all the aforementioned 
methods. The initial ckq in the first iteration of the proposed POTDC method equals to {61 + 92)/ 2 unless 
otherwise is specified. The termination threshold C, for the proposed algorithm is chosen to be equal to 

10-^ 

A. Simulation Example 1 

In this example, the desired and interference sources are locally incoherently scattered with Gaussian 
and uniform angular power densities with central angles of 30° and 10°, respectively. Both sources have 
the same angular spread of 4°. The presumed knowledge of the desired source is different from the actual 
one and is characterized by an incoherently scattered source with Gaussian angular power density whose 
central angle and angular spread are 32° and 1°, respectively. Note that, the presumed knowledge about 
the shape of the angular power density of the desired source is correct while the presumed central angle 
and angular spread deviate from the actual one. 

In Figs. [2] and [3l the output SINR and the objective function values of the problem (ITTI ). respectively, 
are plotted versus SNR. It can be observed from the figures that the proposed new method based on the 
POTDC algorithm has superior performance over the other RABs. Although the method of lITll does not 
have a guaranteed convergence, it results in a better average performance as compared to the method of 
inn and |[T3l . Moreover, the Fig. |3] confirms that the new proposed method archives the global minimum 
of the optimization problem ([TTI ) since the corresponding objective value coincides with the lower-bound 
on the objective function of the problem ([TTT i. Fig. |4] shows the convergence of the iterative POTDC 
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Fig. 2. Example 1: Output SINK versus SNR. 
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Fig. 3. Example 1: Objective function of the problem jilt versus SNR. 



method. It shows the average of the optimal value found by the algorithm over iterations for SNR = 20 
dB. It can be observed that the proposed algorithm converges to the global optimum in about 4 iterations. 
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Fig. 4. Example 1: Objective function of the problem ill) versus the number of iterations. 
B. Simulation Example 2 

In this example, we also consider the locally incoherently scattered desired and interference sources. 
However, compared to the previous example, there is a substantial error in the knowledge of the desired 
source angular power density. 

The interference source is modeled as in the previous example, while the angular power density of the 
desired source is assumed to be a truncated Laplacian function distorted by severe fluctuations. The central 
angle and the scale parameter of the Laplacian distribution is assumed to be 30° and 0.1, respectively, 
and it is assumed to be equal to zero outside of the interval [15°, 45°] as it has been shown in Fig. |5] 
The presumed knowledge of the desired source is different from the actual one and is characterized by 
an incoherently scattered source with Gaussian angular power density whose central angle and angular 
spread are 32° and 1°, respectively. 

Figs. [6] and [T] depict the corresponding output SINR and the objective function values of the problem 
([TT]) obtained by the beamforming methods tested versus SNR. Form these figures, it can be concluded 
that the proposed new method has superior performance over the other methods as well as it achieves 
the global minimum of the optimization problem (ITTI ) 
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Fig. 5. Example 2: Actual and presumed angular power densities of general-rank source. 
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Fig. 6. Example 2: Output SINR versus SNR. 
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Fig. 7. Example 2: Objective function of the problem Jilt versus SNR. 



C. Simulation Example 3 

In this example, we compare the efficiency of the proposed POTDC method to that of the DC iteration- 
based method that can be written for the problem under consideration as 

mill w'^(R + 7l)w 

w 

s.t. /(w('=)) + (v/(wW,w- w('=)))-7?||w||>l (30) 

where the function /(w) = ||Qw|| is replaced with the first two terms of the Taylor expansion of /(w) 
around w^'^). At the first iteration w^^^ is initialized and in the next iterations w^'"') is selected as the 
optimal w obtained from solving (l30l l in the previous iteration. Thus, the iteration are performed over 
the whole vector of variables of the problem. 

The simulation set up is the same as in our Simulation example 1 except that different number of 
antennas are used. For a fair comparison, the initial point qq in the proposed POTDC method and w^^^ 
in (l30l ) are chosen randomly. Table I shows the average number of the iterations for the aforementioned 
methods versus the size of the antenna array. The accuracy is set to 10^^, SNR = — 10 dB, and each 
number in the table is obtained by averaging the results from 200 runs. From this table, it can be seen 
that the number of the iterations for the POTDC method is almost fixed while it increases for the DC- 
iteration method as the size of the array increases. The latter phenomenon can be justified by considering 
the DC iteration-type interpretation of the POTDC method over the one dimensional function of k{a). 
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The dimension of k{a) is independent of the size of the array (thus, the size of the optimization problem), 
while the size of search space for the DC iteration-based method (l30l ). that is, 2M, increases as the size 
of the array increases. The average (over 200 runs) CPU time for the aforementioned methods is also 



TABLE I 

Average number of the iterations 



Array size 


8 


10 


12 


14 


16 


18 


20 


POTDC 


2.940 


2.855 


2.805 


2.835 


2.870 


2.840 


2.920 


DC itera- 
tion 


5.930 


6.925 


7.870 


9.180 


10.430 


11.890 


13.305 



compared in Table 2. Both methods have been implemented in Matlab using CVX software and run 
on the same desktop with Pentium(R) 4 CPU 3.40 GHz. Table 2 confirms that the proposed method is 

TABLE II 

Average CPU time 



Array size 


8 


10 


12 


14 


16 


18 


20 


POTDC 


0.851 


0.867 


0.939 


1.056 


1.153 


1.269 


1.403 


DC itera- 
tion 


4.353 


3.897 


5.882 


5.366 


7.870 


8.575 


10.041 



more efficient that the DC iteration-based one in terms of the time which is spent for solving the same 
problem. Note that although the number of variables in the matrix W of the optimization problem (|27] ) 
is in general A/^ + 1 (since W has to be a Hermitian matrix) after the rank one constraint is relaxed, 
the probability that the optimal W is rank one has been shown to be very high lITOl . Il24l - ll26l . It is also 
approved by this our simulations. Thus, in almost all cases, for different data sets, the actual dimension 
of the problem (|27l ) is 2M + 1. As a result, the average complexity of solving (|27] ) is significantly smaller 
than the worst-case complexity. 

VL Conclusion 

We have considered the RAB problem for general-rank signal model with additional positive semi- 
definite constraint. Such RAB problem corresponds to a non-convex DC optimization problem. We have 
studied this non-convex DC problem rigorously and designed the POTDC algorithm for solving it. It 
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has been proved that the point found by the POTDC algorithm for RAB for general-rank signal model 
with positive semi-definite constraint is a KKT optimal point. Moreover, there is a strong evidence 
that the POTDC method actually finds the globally optimal point of the problem under consideration, 
which is shown in terms of a number of lemmas and one conjecture. Specifically, we have proved a 
number of results that lead us to the equivalence between the claim of global optimality for the POTDC 
algorithm as applied to the problem under consideration and the convexity of the one-dimensional optimal 
value function (l24l) . The latter convexity has been checked numerically by using the convexity on lines 
property of convex functions. The fact that enables such numerical check is that the argument of this 
one-dimensional optimal value function is proved to take values only in a closed interval. The resulted 
RAB method shows superior performance compared to the other existing methods in terms of the output 
SINR. It also has lower average overall complexity than the other traditional methods that can be used 
for the same optimization problem such as, for example, the DC iteration-based method. None of the 
existing methods used for DC programming problems, however, guarantee that the problem can be solved 
in polynomial time. Thus, the fundamental development of this work is that the claim of global optimality 
of the proposed POTDC method boils down to one conjecture that can be easily checked numerically. 
It implies that certain relatively simple DC programming problems (an example of such problem is the 
RAB for general-rank signal model with additional positive semi-definite constraint), which are believed 
to be NP-hard, are likely not NP-hard. 



A. Proof of Lemma 1 

Since the objective function as well as the constraints of the optimization problem (|9]l are all quadratic 
functions of A, this problem is convex. It is easy to verify that this problem satisfies the Slater's constraint 
qualification and as a result the KKT conditions are necessary and sufficient optimality conditions. Let 
us introduce the Lagrangian as 



VII. Appendix 



= w 



(Q^Q + A + A^Q + A^A) w 

+ ^i{\\^f-r,^) 



(31) 
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where ix is the non-negative Lagrange multipUer. The KKT optimaUty conditions are 

VAi(A,^) = (32a) 

||Af < if (32b) 

^i{\\/^f -ri') = Q (32c) 

> 0. (32d) 

where is the vector of zeros. Using the matrix differentiation, the zero gradient condition (I32al i can be 
expressed as Qww^ + Aww^ + /^A = or, equivalently, as 

A = -Qww^(vi^vi^^ + i^lI)-^. (33) 



Moreover, using the matrix inversion lemma, the expression (1331 ) can be simpUfied as 

Qww^ 

^ = - II 112 , • (34) 
||w||^ + 

The Lagrange multiplier can be determined based on the conditions (I32bl )- (l32dl ). For this goal, we find 
a simpler expression for the norm of the matrix Qww^ as follows 

IIQww^f = tr{Qww^ww^Q^} 
= tr{Qww^Q^} • w^w 

= IIQwf ||wf (35) 

Using (l35l) . it can be obtained that 

5{,) 4 IIAII = (36) 
||w||^ + A* 

where the new function 5{ii) is defined for notation simplicity. It is easy to verify that 5{n) is a strictly 
decreasing function with respect to > 0. Consequently, for any arbitrary ^ > 0, it is true that 5{ijl) < 
6{0) = ||Qw^||/||w^||. Depending on whether (5(0) is less than or equal to rj or not, the following two 
cases are possible. If (5(0) < rj, then fi and A can be found as /x = and A = — Qww^/||i(;|p, 
which is obtained by simply substituting /x = in ( [34l ). In this case, the KKT conditions ( |32bh -( f32d|) 
are obviously satisfied. In the other case, when (5(0) > r], the above obtained A for /i = does not 
satisfy the condition ( |32b| ) because ||A|| = (5(0) > rj. Since, 5{fi) is a strictly decreasing function with 
respect to ;U > 0, for satisfying (|32bl l. the value of must be strictly larger than zero and as a result 
the condition (I32cl ) implies that ||A|| = rj. Note that if /^i > and ||A|| obtained by substituting such 
in ( [34t is equal to r], then the KKT conditions <32b|) - f32d|) are all satisfied. Thus, we need to find 
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the value of /x such that the corresponding || A|| is equal to rj. By equating 6{fi) to r], it can be resulted 
that ^0 = ||w||/?y • (||Qw|| — r7||w||). Considering the above two cases together, the optimal A can be 
expressed as 

OWW*' II \ II II 

— V i \ r\ I I I — IT, Qw > 7? W 

'||Qwj|||w|| ' 11^ II - /II II ^27) 
otherwise. 

||w|p ' 

Finally, substituting (l37l) in the objective function of the problem the worst-case signal power for a 
fixed beamforming vector w can be found as shown in (fTOl) . ■ 

B. Proof of Lemma 2 

Let (v^opt, Oopt) denote the optimal solution of the problem (fT2]) . Let us define the following auxiliary 
optimization problem based on the problem 

min w^(R + 7l)w 

w 

s.t. w^Q^Qw = Oopt 

w w < — 5 . (38) 

It can be seen that if w is a feasible point of the problem (|38] ). then the pair {w, Oopt) is also a feasible 
point of the problem ([T2] | which implies that the optimal value of the problem (1381 ) is greater than or equal 
to that of (fT2l) . However, since w^opt is a feasible point of the problem (|38] | and the value of the objective 
function at this feasible point is equal to the optimal value of the problem ([T2] |. i.e., it is equivalent to 
w^^(R + 7l)wopt, it can be concluded that both of the optimization problems (fT2l) and (l38l) have the 
same optimal value. Let us define another auxiliary optimization problem based on the problem (|38] l as 



w 



g= min w^(R + 7l)w 

^opt 



s.t. w^Q^Qw = aopt (39) 



which is obtained from (I38l l by dropping the last constraint of (1381 ). The feasible set of the optimization 
problem (1381 ) is a subset of the feasible set of the optimization problem (|39] l. As a result, the optimal 
value g of the problem (l39l ) is smaller than or equal to the optimal value of the problem (1381 ). and thus 
also, the optimal value of the problem (fT2l ). Using the maxmin theorem |[23l . it is easy to verify that 
9 = Oopt /Amax 1(1^ + 7I)~^Q^Q|- Since g is smaller than or equal to the optimal value of the problem 
([T2l) . it is upper-bounded by Wq^(R + 7l)wo, where wq is an arbitrary feasible point of (fT2] i. The latter 
implies that Oopt < A^ax { (R + 7I)^^Q^Q} • w^(R- + 7l)wo. ■ 
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C. Proof of Lemma 3 

It is easy to verify that the dual problem for both optimization problems of h{a) and k{a) is the same 
and it can be expressed for fixed a as 



max TO 

T,1p 



V'(v^-i)Vr/' 



s.t. (R + 7l)-T-Q^Q + #^0 

V' > (40) 



where r and ijj are the Lagrange multipliers. Based on the dual problem (@0}, a new optimal value 
function is defined as 

d{a) = I maxTQ — -0^^^— ^5 — ^— \ (R- + 7I) 

- tQ^Q + VI ^ 0, V > 0}. (41) 

The optimization problem of k{a) is a convex SDP problem which satisfies the Slater's conditions as 
r = and ^ = 1 is a strictly feasible point for its dual problem (|40] i. Thus, the duality gap between the 
optimization problem of h{a), i.e., the problem (|2TI ) and its dual problem (l40l i is zero. It implies that 

k{a) = d{a), a £[61,62]. (42) 

On the other hand, the optimization problem of h{a) is specifically a quadratically constrained quadratic 
programming (QCQP) problem with only two constraints. It has been recently shown that the duality 
gap between a QCQP in complex variables with two constraints and its dual problem is zero ||27l . ||28l . 
Based on the latter fact, it can be obtained that 

h{a) = d{a), a €[61,62]. (43) 

Using (l42l ) and (1431 ). it can be concluded that the functions h{a) and k{a) are equivalent. 

Let Wq denote the optimal solution of the optimization problem of h{a) which implies that h{a) = 
w^(R + 7l)wa. It is then trivial to verify that = Wq,w^ is a feasible point of the optimization 
problem of k{a) and tr|(R + 7l)Wa| = h{a). Using the fact that h{a) and k{a) are equivalent, it 
can be concluded that tr |(R + 7I)Wq| = k{a) which implies that is the optimal solution of the 
optimization problem of k{a). The latter means that, for a fixed value of a, the optimization problem 
of k{a) has always a rank-one solution. A method for extracting a rank-one solution from a general- 
rank solution of QCQP is explained, for example, in ll28l . It is trivial to see that the scaled dominant 
eigenvector of such a rank-one solution of the optimization problem of k{a) is the optimal solution of 
the optimization problem of h{a). ■ 
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D. Proof of Lemma 4 

First, we prove that l{a, ac) is a convex function with respect to a. For this goal, let W^^ and W^^ 
denote the optimal solution of the optimization problems of /(ai,Qc) and l{a2^ac), respectively, i.e., 
l{ai,ac) = tr I (R + 7I)Wqj I and l{a2,ac) = tr |(R + 7l)Wa2|, where ai and 02 are any two 
arbitrary points in the interval [^1,^2]- It is trivial to verify that ^W^^ + (1 — 9)'Wa2 is a feasible point 
of the corresponding optimization problem of l{9ai + (1 — 0)a2^ oic) (see the definition (|28])). Therefore, 

l{eai +{l-e)a2,ac) 

< tr{(R+7l)(0W„,+ (l-0)W,J} 

= 0tr{(R + 7l)W,J 

+(l-^)tr{(R + 7l)W,J 

= ei{ai,ac) + (1 - 0)l{a2, Oc) (44) 

which proves that l{a,ac) is a convex function with respect to a. 

In order to show that l{a,ac) is greater than or equal to k{a), it suffices to show that the feasible 
set of the optimization problem of l{a,ac) is a subset of the feasible set of the optimization problem of 
k{a). Let Wi denote a feasible point of the optimization problem of l{a,ac), it is easy to verify that 
Wi is also a feasible point of the optimization problem of k{a) if the inequality ^/a < ^JoT^ + f^§* 
holds. This inequality can be rearranged as 

(V^-V^)2>0 (45) 

and it is valid for any arbitrary a. Therefore, Wi is also a feasible point of the optimization problem of 
k{a) which implies that /(a,ac) > k(a). 

In order to show that the right and left derivatives are equal, we use the result of ||29l Theorem 10] 
which gives expressions for the directional derivatives of a parametric SDP. Specifically, in ||29l Theorem 
10] the directional derivatives for the following optimal value function 

^(u) ^ {mill /(y, u) I G(y, u) < 0„xn} (46) 
y 

are derived, where /(y, u) and G(y, u) are a scaler and an n x n matrix, respectively, y G TU^ is 
the optimization variables and u G is the optimization parameters. Let Uc be an arbitrary fixed 
point. If the optimization problem of V'(uc) poses certain properties, then according to ||29] Theorem 
10] it is directionally differentiable at Uc. These properties are (i) the functions /(y, u) and G(y, u) 
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are continuously differentiable, (ii) the optimization problem of ip{uc) is convex, (iii) the set of optimal 
solutions of the optimization problem of ipiuc) denoted as A4 is nonempty and bounded, (iv) the Slater 
condition for the optimization problem of V'(uc) holds true, and (v) the inf-compatness condition is 
satisfied. Here inf-compatness condition refers to the condition of the existence of a > ^(uc) and a 
compact set S C TV^ such that {y|/(y,u) < a,G(y, u) ^ 0} C 5" for all u in a neighborhood of 
Uc- If for all u the optimization problem of V'(u) is convex and the set of optimal solutions of ip{u) is 
non-empty and bounded, then the inf-compactness conditions holds automatically. 
The directional derivative of ^(u) at Uc in a direction d G TV' is given by 



■(/''(ucd) = min maxd"^Vu^(y, ^2, Uc), 



(47) 



where Z is the set of optimal solutions of the dual problem of the optimization problem of V'(uc) and 
L(y, ri, u) denotes the Lagrangian defined as 

L(y, u) ^ /(y, u) + tr (fi • G(y, u)) (48) 

where f2 denotes the Lagrange multiplier matrix. 

Let us look again to the definitions of the optimal value functions k{a) and l{a,ac) (l24l ) and ( [28] ). 
respectively, and define the following block diagonal matrix 



Gi(W,a)= (49) 
/-W ^ 

r/2tr{W}-(Va-l)^ 
tr{Q^QW} -a 

\Q a-tr{Q^QW}y 

as well as another block diagonal matrix denoted as G2(W, a) which has exactly same structure as the 
matrix Gi(W, a) with only difference that the element ry^trjW} — (i/a — 1)^ in Gi(W, a) is replaced 
by jf ■ tr{W} + (-y/a^ — 1) + a [l/^/a^— l) in G2(W, a). Then the optimal value functions k{a) and 
/(a, Oc) can be equivalently recast as 



k{a) 



and 



l{a, Uc 



imntr |(R + 7l) • w} |Gi(W,a)^0 



|imntr|(R + 7l) • w} |G2(W, a) ^ |> , 
6*1 < a < di- 



(50) 



(51) 
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It is trivial to verify that the optimization problems of k{ac) and /(oc, etc) can be expressed as 

mill tr|(R + 7l)-W| 
s.t. tr{Q^QW} = Oc 



tr{W}< 



^2 



W ^ 0. (52) 

The problem (|52] | is convex and its solution set is non-empty and bounded. Indeed, let Wi and W2 
denote two optimal solutions of the problem above. The Euclidean distance between Wi and W2 can 
be expressed as 

d(Wi, W2) = II Wi - W2II 



tr{Wf } + tr{W|} - 2tr{WiW2} 



< (53) 

where the last line is due to the fact that the matrix product Wi W2 is positive semi-definite and, therefore, 
tr{WiW2} > 0, and also the fact that for any arbitrary positive semi-definite matrix tr{A^} < tr{A}^. 
From the equation above, it can seen that the distance between any two arbitrary optimal solutions of 
( l52l) is finite and, therefore, the solution set is bounded. As it was mentioned in the proof of Lemma 2, 
the optimization problem (|52l ) satisfies the strong duality. In a similar way, it can be shown that the 
inf-compactness condition is satisfied by verifying that the optimization problems of k{a) and l{a,ac) 
are convex and their corresponding solution sets are bounded for any a. Therefore, both of the optimal 
value functions k{a) and l{a,ac) are directionally differentiable at ac- 

Using the result of ||29] Theorem 10], the directional derivatives of k{a) and l{a, ac) can be respectively 
computed as 

/c'(a,d) = min maxd I trjn • -^Gi(W,a)| M (54) 

and 

l'(a,ac.,d) = min maxd f tr|fJ • -^G2(W, q)| | | (55) 

^ ' ' ^ weMnez \ ^ da ^ >\a=aj j 

where M and Z denote the optimal solution sets of the optimization problem of (l52l ) and its dual 
problem, respectively. Using the definitions of Gi(W,a) and G2(W,a), it can be seen that the terms 
dGiiyV ,a) / da and dGi{yV ,a) / da are equal at a = ac and, therefore, the directional derivatives are 
equivalent. The latter implies that the left and right derivatives of k{a) and l{a, Uc) are equal at a = ac- 
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E. Proof of Lemma 5 

i) As it has been explained, the optimization problem in Algorithm 1 at iteration i, i > 2 is obtained 
by linearizing ^/o^ at aopt,i-i- Since Wopt,i~i and aopt,i~i are feasible for the optimization problem at 
iteration i, it can be straightforwardly concluded that the optimal value of the objective at iteration 
i is less than or equal to the optimal value at the previous iteration, i.e., tr |(R + 7l)Wopt j| < 
tr |(R + 7l)Wopt^j_i| which completes the proof. 

ii) Since the sequence of the optimal values, i.e., tr |(R + 7l)Wopt,t| ,z > 1 is non-increasing and 
bounded from below (every optimal value is non-negative), the sequence of the optimal values converges. 

iii) The proof follows straightforwardly from Proposition 3.2 of 1301 Section 3]. ■ 
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